A log file is an extremely valuable piece of information which is provided by a server. Almost all servers, services, and applications provide some sort of logging. But what is a log file? A log file records events and actions that take place during the runtime of a service or application.
So why are log files so important? Log files provide us with a precise view of the behavior of a server as well as critical information about when, how and “by whom” a server is being accessed. This kind of information can help us monitor the performance, troubleshoot and debug applications, as well as help forensic investigators unfold the chain of events that may have led to a malicious activity.
Let’s take as an example a web server. Most commonly, Apache HTTP Server will provide two main log files – access.log and the error.log. The access.log records all requests for files. If a visitor requests www.example.com/main.php, the following entry will be added in the log file.
|
|
The above log describes that a visitor with an IP address of 88.54.124.178 requested main.php file on April 16th, 2016 07:44 and the request was successful.
This information might not be too interesting, but what if the log file described that a visitor with IP 88.54.124.178 requested dump_database.php file on April 16th, 2016 07:44 and the request was successful?
In the absence of that log file, you might have never known that someone discovered and ran a secret or restricted script you have on your website that dumps the database.
Having established that a log file is a critical asset, let’s look at an everyday example of how a log file would help identify when, how and “by whom”a website was hacked.
Investigation
Let’s assume that a website we administer got defaced. Let’s also assume that the site was a simple and up-to-date WordPress website running on a fully patched Ubuntu Server.
After reaching out for help, the forensic team took the server “offline” in order to be able to proceed with the investigation.
Isolating the server is done to preserve the current state of the system and its logs, block remote access to the attacker (in the case a backdoor was installed), as well as prevent interaction with any other network machines.
In order to fulfill the scope of the investigation, which is to identify malicious activity on the web server, the methodology would require the creation of a forensically sound copy of the server and then proceed with the investigation, however, since there are no plans to pursue legal action against the attacker, the forensic team can work on the original data.
Evidence to Look for in an Investigation
In order to start an investigation, the investigator needs to identify what evidence to look for. Usually, evidence of an attack involves direct access to “hidden” or unusual files, access to the administration area with or without authentication, remote code execution, SQL injection, file inclusion, cross-site scripting (XSS) and other unusual behavior that might indicate vulnerability scanning or reconnaissance.
Let us assume that for our example, the web-server’s access.log is available.
|
|
The access.log tends to be quite a large file, often containing thousands of recorded requests.
|
|
Checking every single line would be impractical, so what we’ll want to do is to filter out data that would most probably be of no interest. That usually includes resources such as images and CSS stylesheets. Some investigators also prefer to strip out JavaScript files too.
In this case, however, since the website is running WordPress, we will use a slightly different approach. Instead of ruling out some data, we will filter access.log for WordPress-specific characteristics.
|
|
The above command, will filter access.log and show only records that contain strings containing wp-admin which is the default administration folder of WordPress, wp-login which is part of the login file of WordPress (wp-login.php) and finally, POST which will show HTTP requests sent to the server using the POST method, which are most likely login form submissions.
The output returns a number of results. After sifting through them, we’ll concentrate on the following single record:
|
|
We see that the IP ++84.55.41.57++ accessed the WordPress administration successfully.
Let’s see what else the user with this IP address did. We’ll use grep once again to filter the access.log with that IP.
|
|
This results in the following interesting records.
|
|
Let’s analyze these records a bit further.
The attacker accessed the login screen.
|
|
The attacker submitted the login form (HTTP request using the POST method) and was redirected (302 HTTP status code).
|
|
The attacker was redirected to wp-admin (the WordPress dashboard) which means authentication was successful.
|
|
The attacker navigated to the theme editor.
|
|
The attacker tried to edit file 404.php, which is a very common tactic used to inject malicious code into the file. The attacker most probably failed in doing so due to a lack of write permissions.
|
|
The attacker accessed the plugin installer.
|
|
The attacker installed and activated the file-manager plugin.
|
|
The attacker used the file-manager plugin to upload r57.php, which is a PHP webshell script.
|
|
The log indicates that the attacker ran an r57 shell script. The query strings ?1 (the attacker ran phpinfo();) and ?28 (the attacker got a list of services) indicate navigation through the different sections of the shell script. It appears that he didn’t find anything interesting.
|
|
The attacker’s last action was to edit the index file of the theme through the file-manager plugin and replaced its contents with the word “HACKED!”
|
|
Based on the above information, we now have a timeline of the attacker’s actions that led to the defacement of the website. However, there is a missing piece in the puzzle. How did the attacker get the login credentials in the first place?
Assuming that we are certain that the administrator password was not leaked or bruteforced, let’s go back and see if we can find anything regarding this matter
The current access.log did not contain any clues on what might have happened. However, there is more than just the one access.log file we were investigating. Apache HTTP Server’s log rotation, archived old log files. Listing the /var/log/apache2/ directory lists 4 additional log files. Let’s investigate.
Firstly, we’ll filter the logs to see if any actions were taken by IP 84.55.41.57. One of the logs was bombarded with records that clearly indicate a SQL injection attack on what seems to be a custom plugin.
|
|
Let’s assume that this plugin was created by copy-and-pasting some which the system administrator found online. The script was meant to check for a user’s validity based on a given ID. The plugin had a form exposed on the main page of the website which was sending an AJAX GET request to /wordpress/wp-content/plugins/custom_plugin/check_user.php.
By analyzing check_user.php it is immediately obvious that the script is poorly written and vulnerable to a SQL injection attack.
|
|
The amount of records in the access.log and the pattern indicate that the attacker used a SQL injection exploitation tool to exploit the SQL injection vulnerability.
We will not dig deeper into the SQL injection attack, or how to fix SQL injection vulnerabilities as this is outside the scope of this article, however, the records in the log would resemble the following.
|
|
The above is a very strong indication that the WordPress database has been compromised and that any data in that database has potentially been stolen.
Analysis
Through this investigation, we can now create the chain of events that have led to this attack.
Some questions still remain, such as who was behind the attack. At this point, it is only possible to know the attacker’s IP address. It is very difficult, and probably infeasible, to attempt to attribute most attacks unless the attacker left concrete evidence that ties to a real person’s identity. Bear in mind that attackers frequently make use of proxies and anonymity networks such as Tor to conduct most attacks in order to mask their real location.
The bottom line is that unsafe code that led to a SQL injection attack was present in a custom WordPress plugin. Had the site been tested for security vulnerabilities before being deployed in a production environment, it would not have been possible for the attacker to take advantage of the security vulnerability which caused the defacement.
The attacker of the above fictitious example was very sloppy and left a significant amount of evidence and tracks which would have aided an investigator and made the investigation very easy. This, however, is not always the case, especially when dealing with more sophisticated attacks.
Refere：
https://dzone.com/articles/using-logs-to-investigate-a-web-application-attack
https://www.acunetix.com/blog/articles/using-logs-to-investigate-a-web-application-attack/